Skip to main content

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Brak Date: Jan 30, 2005 2:31am
Forum: web Subject: Re: Full-text search of the Wayback archive


Now that's a fascinating idea. It's not the reason it was taken down. We hope to have that functionality available again.

Doing a distributed index sounds like a worthy p2p project for all sorts of reasons. The problem with projects of this scale might be the sheer size of the data. If the index were, let's say, 25 Terabytes, then if each user donated 20GB of hard drive space, it would take, in theory, a minimum of 1250 users, and more realistically 5000 users (taking into account redundancy and load sharing). That's a taller order than most p2p apps. Anything is possible, however.

For now, stay tuned...